Ar/vr navigation with authentication using an integrated scrollwheel and fingerprint sensor user input apparatus

ABSTRACT

A virtual reality (VR) or augmented reality (AR) system includes: a display configured to display a user interface to a user; an integrated scrollwheel and fingerprint sensor (FPS) user input apparatus, comprising a scrollwheel configured to detect a rotational navigation input from the user and an FPS configured to detect a biometric input from the user; and a processing system configured to: receive the rotational navigation input via the scrollwheel of the integrated scrollwheel and FPS user input apparatus; update a displayed user interface on the display based on the received rotational navigation input, wherein updating the displayed user interface comprises updating a user selection on a displayed menu; receive an activation input for the updated user selection on the displayed menu via the FPS of the integrated scrollwheel and FPS user input apparatus; and execute an operation corresponding to the updated user selection on the displayed menu.

BACKGROUND

Input devices, including touch sensors and fingerprint sensors, arewidely used in a variety of electronic systems. Touch sensors andfingerprint sensors may include a sensing region, often demarked by asurface, in which the sensor determines the presence, location, motion,and/or features of one or more input objects. Touch sensors andfingerprint sensors may be used to provide interfaces for the electronicsystem. For example, touch sensors and fingerprint sensors may be usedas input devices for larger computing systems (e.g., opaque touchpadsand fingerprint sensors integrated in, or peripheral to, notebook ordesktop computers). Touch sensors and fingerprint sensors are also oftenused in smaller computing systems (e.g., touch screens and fingerprintsensors integrated in smartphones).

In recent years, augmented reality (AR) and virtual reality (VR) systemsare becoming increasingly popular, and there may be use cases for touchsensors and fingerprint sensors in AR and VR systems. However,conventional touch sensor and fingerprint sensor devices are oftenunsuitable for AR and VR environments, particularly where a user may notbe able to conveniently look at or see the user input device that theuser is interacting with.

SUMMARY

In an exemplary embodiment, the present disclosure provides a virtualreality (VR) or augmented reality (AR) system. The system includes: adisplay configured to display a user interface to a user of the VR or ARsystem; an integrated scrollwheel and fingerprint sensor (FPS) userinput apparatus, comprising a scrollwheel configured to detect arotational navigation input from the user and an FPS configured todetect a biometric input from the user; and a processing systemconfigured to: receive the rotational navigation input via thescrollwheel of the integrated scrollwheel and FPS user input apparatus;update a displayed user interface on the display based on the receivedrotational navigation input, wherein updating the displayed userinterface comprises updating a user selection on a displayed menu;receive an activation input for the updated user selection on thedisplayed menu via the FPS of the integrated scrollwheel and FPS userinput apparatus; and execute an operation corresponding to the updateduser selection on the displayed menu.

In a further exemplary embodiment, the processing system is furtherconfigured to: receive the biometric input from the FPS of theintegrated scrollwheel and FPS user input apparatus; performauthentication based on the received biometric input; and execute theoperation corresponding to the updated user selection on the displayedmenu in response to successful authentication.

In a further exemplary embodiment, performing authentication based onthe received biometric input comprises determining whether the receivedbiometric input matches a stored template of a pre-enrolled authorizeduser.

In a further exemplary embodiment, the processing system is furtherconfigured to: receive another activation input for another userselection on the displayed menu via the FPS of the integratedscrollwheel and FPS user input apparatus; and execute another operationcorresponding to the another user selection on the displayed menu inresponse to the another activation input without performingauthorization for the another operation.

In a further exemplary embodiment, the integrated scrollwheel and FPSuser input apparatus and the display are both physically integrated intoa same housing.

In a further exemplary embodiment, the integrated scrollwheel and FPSuser input apparatus and the display are both physically integrated intoa housing of a VR or AR headset.

In a further exemplary embodiment, the integrated scrollwheel and FPSuser input apparatus and the display are physically integrated intoseparate housings of separate devices.

In another exemplary embodiment, the present disclosure provides amethod for navigation. The method includes: displaying, by a display, auser interface to a user; receiving, by a processing system, via ascrollwheel of an integrated scrollwheel and fingerprint sensor (FPS)user input apparatus, a rotational navigation input; updating, on thedisplay, the displayed user interface based on the received rotationalnavigation input, wherein updating the displayed user interfacecomprises updating a user selection on a displayed menu; receiving, by aprocessing system, an activation input for the updated user selection onthe displayed menu; and executing, by a processing system, an operationcorresponding to the updated user selection on the displayed menu.

In a further exemplary embodiment, the activation input is received viaan FPS of the integrated scrollwheel and fingerprint sensor (FPS) userinput apparatus.

In a further exemplary embodiment, the method further includes:receiving, via the FPS of the integrated scrollwheel and FPS user inputapparatus, a biometric input; and performing, by the processing system,authentication based on the received biometric input; wherein executingthe operation corresponding to the updated user selection on thedisplayed menu is in response to successful authentication.

In a further exemplary embodiment, performing authentication based onthe received biometric input comprises determining whether the receivedbiometric input matches a stored template of a pre-enrolled authorizeduser.

In a further exemplary embodiment, the method further includes:receiving, by the processing system, another activation input foranother user selection on the displayed menu via the FPS of theintegrated scrollwheel and FPS user input apparatus; and executing, bythe processing system, another operation corresponding to the anotheruser selection on the displayed menu in response to the anotheractivation input without performing authorization for the anotheroperation.

In a further exemplary embodiment, the integrated scrollwheel and FPSuser input apparatus and the display are both physically integrated intoa same housing.

In a further exemplary embodiment, the integrated scrollwheel and FPSuser input apparatus and the display are both physically integrated intoa housing of a VR or AR headset.

In a further exemplary embodiment, the integrated scrollwheel and FPSuser input apparatus and the display are physically integrated intoseparate housings of separate devices.

In yet another exemplary embodiment, the present disclosure provides anintegrated scrollwheel and fingerprint sensor (FPS) user inputapparatus. The apparatus includes: a scrollwheel configured to detect arotational navigation input from a user, wherein the rotationalnavigation input from the user corresponds to a clockwise orcounterclockwise motion of a user's finger on the scrollwheel, andwherein the scrollwheel has an outer boundary and an inner boundary,wherein the inner boundary defines an area surrounded by thescrollwheel; and an FPS, proximate to the scrollwheel and disposedwithin the area surrounded by the scrollwheel, configured to detect abiometric input from the user.

In a further exemplary embodiment, the apparatus further includes: atactile ridge disposed between the scrollwheel and the FPS.

In a further exemplary embodiment, the FPS is further configured todetect an activation input from the user.

In a further exemplary embodiment, detecting the activation input fromthe user is based on detecting the presence of the user's finger on theFPS or detecting a force applied on the FPS by the user's finger.

In a further exemplary embodiment, the FPS comprises an integratedbutton, and wherein detecting the activation input from the user isbased on the integrated button being pressed.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1A-1B are schematic block diagrams of exemplary input devices.

FIGS. 2A-2C are block diagrams depicting exemplary AR or VRenvironments.

FIG. 3 is a schematic diagram illustrating an integrated scrollwheel andFPS user input apparatus in an exemplary embodiment.

FIGS. 4A-4B depict exemplary manners of utilizing the integratedscrollwheel and FPS user input apparatus depicted in FIG. 3 .

FIG. 5 is a flowchart depicting an exemplary process for utilizing anintegrated scrollwheel and FPS user input apparatus in accordance withan exemplary embodiment.

DETAILED DESCRIPTION

The following detailed description is exemplary in nature and is notintended to limit the disclosure or the application and uses of thedisclosure. Furthermore, there is no intention to be bound by anyexpressed or implied theory presented in the preceding background andbrief description of the drawings, or the following detaileddescription.

Exemplary embodiments of the present disclosure provide for devices andmethods for navigation with seamless authentication in an AR or VRenvironment using an integrated scrollwheel and fingerprint sensor (FPS)user input apparatus. In various embodiments, by positioning ascrollwheel around a fingerprint sensor, a user-friendly and intuitivemanner of receiving various forms of user input to perform navigationwith authentication is achieved, even in situations (such as AR and VR)where the user may not be able to look at or see the input device thatthe user is interacting with.

FIGS. 1A-1B illustrate example input devices for providing an exemplarydescription of how touch sensors and fingerprint sensors may beconfigured in accordance with exemplary embodiments of the presentdisclosure. It will be appreciated that FIGS. 1A-1B are provided by wayof example, and that the types of touch sensors and fingerprint sensorsusable in exemplary embodiments of the present disclosure are notlimited to the examples discussed herein in connection with FIGS. 1A-1B.

FIG. 1A is a block diagram depicting an example input device 100 withinwhich the present embodiments may be implemented. The input device 100may be configured to provide input to an electronic system (not shownfor simplicity). As used in this document, the term “electronic system”(or “electronic device”) broadly refers to any system capable ofelectronically processing information. Examples of electronic systemsinclude personal computing devices (e.g., desktop computers, laptopcomputers, netbook computers, tablets, web browsers, e-book readers, andpersonal digital assistants (PDAs)), wearable computers (e.g., smartwatches and activity tracker devices), composite input devices (e.g.,physical keyboards, joysticks, and key switches), data input devices(e.g., remote controls and mice), data output devices (e.g., displayscreens and printers), remote terminals, kiosks, video game machines(e.g., video game consoles, portable gaming devices, and the like),communication devices (e.g., cellular phones, such as smart phones), andmedia devices (e.g., recorders, editors, and players such astelevisions, set-top boxes, music players, digital photo frames, anddigital cameras). Additionally, the electronic system may be a host or aslave to the input device 100.

The input device 100 may be implemented as a physical part of theelectronic system, or may be physically separate from the electronicsystem. The input device 100 may be coupled to (and communicate with)components of the electronic system using wired or wirelessinterconnections and communication technologies, such as buses andnetworks. Example technologies may include Inter-Integrated Circuit(I2C), Serial Peripheral Interface (SPI), Personal System/2 (PS/2),Universal Serial Bus (USB), Bluetooth®, Infrared Data Association(IRDA), and various radio frequency (RF) communication protocols definedby the IEEE 802.11 or other standards.

In the example of FIG. 1A, input device 100 includes a sensor 105. Thesensor 105 comprises one or more sensing elements configured to senseinput provided by one or more input objects in a sensing region of theinput device 100. Examples of input objects include fingers, styli, andhands. The sensing region may encompass any space above, around, in,and/or proximate to the sensor 105 in which the input device 100 is ableto detect user input (e.g., user input provided by one or more inputobjects). The sizes, shapes, and/or locations of particular sensingregions (e.g., relative to the electronic system) may vary depending onactual implementations. In some embodiments, the sensing region mayextend from a surface of the input device 100 in one or more directionsinto space, for example, until a signal-to-noise ratio (SNR) of thesensors fall below a threshold suitable for accurate object detection.For example, the distance to which this sensing region extends in aparticular direction may be on the order of less than a millimeter,millimeters, centimeters, or more, and may vary significantly with thetype of sensing technology used and/or the accuracy desired. In someembodiments, the sensor 105 may detect input involving no physicalcontact with any surfaces of the input device 100, contact with an inputsurface (e.g., a touch surface and/or screen) of the input device 100,contact with an input surface of the input device 100 coupled with someamount of applied force or pressure, and/or a combination thereof. Invarious embodiments, input surfaces may be provided by surfaces ofsensor substrates within which or on which sensor elements arepositioned, or by face sheets or other cover layers positioned oversensor elements.

The input device 100 comprises one or more sensing elements fordetecting user input. Some implementations utilize arrays or otherregular or irregular patterns of sensing elements to detect the inputobject. The input device 100 may utilize different combinations ofsensor components and sensing technologies to detect user input in thesensing region.

The input device 100 may utilize various sensing technologies to detectuser input. Example sensing technologies may include capacitive,elastive, resistive, inductive, magnetic, acoustic, ultrasonic, andoptical sensing technologies. In some embodiments, the input device 100may utilize capacitive sensing technologies to detect user inputs. Forexample, the sensing region may include one or more capacitive sensingelements (e.g., sensor electrodes) to create an electric field. Theinput device 100 may detect inputs based on changes in capacitance ofthe sensor electrodes. For example, an object in contact with (or closeproximity to) the electric field may cause changes in the voltage and/orcurrent in the sensor electrodes. Such changes in voltage and/or currentmay be detected as “signals” indicative of user input.

The sensor elements may be arranged in arrays (regular or irregularpatterns) or other configurations to detect inputs. In someimplementations, separate sensing elements may be ohmically shortedtogether to form larger sensor electrodes. Some capacitive sensingimplementations may utilize resistive sheets that provide a uniformresistance.

Example capacitive sensing technologies may be based on“self-capacitance” (also referred to as “absolute capacitance”) and/or“mutual capacitance” (also referred to as “transcapacitance”).Transcapacitance sensing methods detect changes in the capacitivecoupling between sensor electrodes. For example, an input object nearthe sensor electrodes may alter the electric field between the sensorelectrodes, thus changing the measured capacitive coupling of the sensorelectrodes. In some embodiments, the input device 100 may implementtranscapacitance sensing by detecting the capacitive coupling betweenone or more transmitter sensor electrodes (also “transmitter electrodes”or “drive electrodes”) and one or more receiver sensor electrodes (also“receiver electrodes” or “pickup electrodes”). For example, transmittersensor electrodes may be modulated relative to a reference voltage totransmit transmitter signals while receiver sensor electrodes may beheld at a relatively constant voltage to receive the transmittedsignals. The reference voltage may be, for example, a substantiallyconstant voltage or system ground. In some embodiments, transmittersensor electrodes and receiver sensor electrodes may both be modulated.The signals received by the receiver sensor electrodes may be affectedby environmental interference (e.g., from other electromagnetic signalsand/or objects in contact with, or in close proximity to, the sensorelectrodes). Sensor electrodes may be dedicated transmitters orreceivers, or may be configured to both transmit and receive.

In some implementations, the input device 100 is configured to provideimages that span one, two, three, or higher dimensional spaces. Theinput device 100 may have a sensor resolution that varies fromembodiment to embodiment depending on factors such as the particularsensing technology involved and/or the scale of information of interest.In some embodiments, the sensor resolution is determined by the physicalarrangement of an array of sensing elements, where smaller sensingelements and/or a smaller pitch can be used to define a higher sensorresolution.

The input device 100 may be implemented as a fingerprint sensor having asensor resolution high enough to capture discriminative features of afingerprint. In some implementations, the fingerprint sensor has aresolution sufficient to capture minutia (including ridge endings andbifurcations), orientation fields (sometimes referred to as “ridgeflows”), and/or ridge skeletons. These are sometimes referred to aslevel 1 and level 2 features, and in an exemplary embodiment, aresolution of at least 250 pixels per inch (ppi) is capable of reliablycapturing these features. In some implementations, the fingerprintsensor has a resolution sufficient to capture higher level features,such as sweat pores or edge contours (i.e., shapes of the edges ofindividual ridges). These are sometimes referred to as level 3 features,and in an exemplary embodiment, a resolution of at least 750 pixels perinch (ppi) is capable of reliably capturing these higher level features.In silicon fingerprint sensors, a resolution of around 500 ppi may beused to balance cost and performance, but it will be appreciated that aresolution of 5000 ppi (corresponding to 5.08 μm pitch) or more can beachieved in silicon fingerprint sensors.

In some embodiments, a fingerprint sensor is implemented as a placementsensor (also “area” sensor or “static” sensor) or a swipe sensor (also“slide” sensor or “sweep” sensor). In a placement sensor implementation,the sensor is configured to capture a fingerprint input as the user'sfinger is held stationary over the sensing region. Typically, theplacement sensor includes a two dimensional array of sensing elementscapable of capturing a desired area of the fingerprint in a singleframe. In a swipe sensor implementation, the sensor is configured tocapture a fingerprint input based on relative movement between theuser's finger and the sensing region. In some embodiments, the swipesensor may include a linear array or a thin two-dimensional array ofsensing elements configured to capture multiple frames as the user'sfinger is swiped or moves over the sensing region. The multiple framesmay then be reconstructed to form an image of the fingerprintcorresponding to the fingerprint input. In some implementations, thesensor is configured to capture both placement and swipe inputs.

In some embodiments, a fingerprint sensor is configured to capture lessthan a full area of a user's fingerprint in a single user input(referred to herein as a “partial” fingerprint sensor). Typically, theresulting partial area of the fingerprint captured by the partialfingerprint sensor is sufficient for the system to perform fingerprintmatching from a single user input of the fingerprint (e.g., a singlefinger placement or a single finger swipe). Some exemplary imaging areasfor partial placement sensors include an imaging area of 100 mm² orless. In another exemplary embodiment, a partial placement sensor has animaging area in the range of 20-50 mm². In some implementations, thepartial fingerprint sensor has an input surface that is of the same orsubstantially the same size as the imaging area.

In FIG. 1A, a processing system 110 is included with the input device100. The processing system 110 may comprise parts of or all of one ormore integrated circuits (ICs) and/or other circuitry components. Theprocessing system 110 is coupled to the sensor 105, and is configured tooperate hardware of the input device 100 (e.g., sensing hardware of thesensor 105) to detect input in the sensing region.

The processing system 110 may include driver circuitry configured todrive sensing signals with sensing hardware of the input device 100and/or receiver circuitry configured to receive resulting signals withthe sensing hardware. For example, processing system 100 may beconfigured to drive transmitter signals onto transmitter sensorelectrodes of the sensor 105, and/or receive resulting signals detectedvia receiver sensor electrodes of the sensor 105.

The processing system 110 may include a non-transitory computer-readablemedium having processor-executable instructions (such as firmware code,software code, and/or the like) stored thereon. The processing system110 can be implemented as a physical part of the sensor 105, or can bephysically separate from the sensor 105. Also, constituent components ofthe processing system 110 may be located together, or may be locatedphysically separate from each other. For example, the input device 100may be a peripheral device coupled to a computing device, and theprocessing system 110 may comprise software configured to run on acentral processing unit of the computing device and one or more ICs(e.g., with associated firmware) separate from the central processingunit. As another example, the input device 100 may be physicallyintegrated in a mobile device, and the processing system 110 maycomprise circuits and firmware that are part of a main processor of themobile device. The processing system 110 may be dedicated toimplementing the input device 100, or may perform other functions, suchas operating display screens, driving haptic actuators, etc.

The processing system 110 may operate the sensing element(s) of thesensor 105 of the input device 100 to produce electrical signalsindicative of input (or lack of input) in a sensing region. Theprocessing system 110 may perform any appropriate amount of processingon the electrical signals to translate or generate the informationprovided to the electronic system. For example, the processing system110 may digitize analog electrical signals received via the sensorelectrodes and/or perform filtering or conditioning on the receivedsignals. In some embodiments, the processing system 110 may subtract orotherwise account for a baseline associated with the sensor electrodes.For example, the baseline may represent a state of the sensor electrodewhen no user input is detected. Accordingly, the information provided bythe processing system 110 to the electronic system may reflect adifference between the signals received from the sensor electrodes and abaseline associated with each sensor electrode. As yet further examples,the processing system 110 may determine positional information,recognize inputs as commands, recognize handwriting, match biometricsamples, and the like.

In some embodiments, the input device 100 may include a touch screeninterface (e.g., display screen), as well as a fingerprint sensor,wherein a sensing region of the fingerprint sensor at least partiallyoverlaps a sensing region of the touch screen interface. The displaydevice may be any suitable type of dynamic display capable of displayinga visual interface to a user, including an inorganic light-emittingdiode (LED) display, organic LED (OLED) display, cathode ray tube (CRT),liquid crystal display (LCD), plasma display, electroluminescence (EL)display, or other display technology. The display may be flexible orrigid, and may be flat, curved, or have other geometries. The displaymay include a glass or plastic substrate for thin-film transistor (TFT)circuitry, which may be used to address display pixels for providingvisual information and/or providing other functionality. The displaydevice may include a cover lens (sometimes referred to as a “coverglass”) disposed above display circuitry and above inner layers of thedisplay module, and the cover lens may also provide an input surface forthe input device 100. Examples of cover lens materials include opticallyclear amorphous solids, such as chemically hardened glass, and opticallyclear crystalline structures, such as sapphire. The input device 100 andthe display device may share physical elements. For example, some of thesame electrical components may be utilized for both displaying visualinformation and for input sensing with the input device 100, such asusing one or more display electrodes for both display updating and inputsensing. As another example, the display screen may be operated in partor in total by the processing system 110 in communication with the inputdevice 100.

FIG. 1B is a block diagram depicting the input device 100 as including afingerprint sensor 105 b. The fingerprint sensor 105 b is configured tocapture an image of the fingerprint from a finger 140. The fingerprintsensor 105 b is disposed underneath a cover layer 112 that provides aninput surface for the fingerprint to be placed on or swiped over thefingerprint sensor 105 b. The sensing region 120 may include an inputsurface with an area larger than, smaller than, or similar in size to afull fingerprint. The fingerprint sensor 105 b has an array of sensingelements with a resolution configured to detect surface variations ofthe finger 140. In certain embodiments, the fingerprint sensor 105 b maybe disposed within the active area of a display.

FIG. 2A is a block diagram depicting an exemplary AR or VR environment200 a in which exemplary embodiments of the present disclosure may beutilized. The environment 200 a includes, for example, an AR or VRheadset 230 (e.g., in the form of goggles worn on a user's head whichdisplays VR or AR information to the user). In this embodiment, the ARor VR headset 230 may be self-contained and have integrated processingcapabilities for generating a display and receiving user input (e.g.,through user interface elements integrated into the AR or VR headset230, such as manual controls which can be actuated by a user or throughsensor-based detection of user gestures).

In an exemplary embodiment, the AR or VR headset 230 may include a touchsensor and/or a fingerprint sensor, each having a respective processingsystem as discussed above with respect to FIGS. 1A-1B implemented in theAR or VR headset 230. A combined processing system which controls both atouch sensor and a fingerprint sensor may also be utilized.

FIG. 2B is a block diagram depicting another exemplary AR or VRenvironment 200 b in which exemplary embodiments of the presentdisclosure may be utilized. The environment 200 b includes, for example,a host device 210 in addition to an AR or VR headset 230 (e.g., in theform of goggles worn on a user's head which displays VR or ARinformation to the user). In this embodiment, the AR or VR headset 230may communicate with the host device and carry out various processingoperations with the assistance of the host device 210—for example,relating to generating a display for the user and processing receiveduser inputs (e.g., received through user interface elements integratedinto the AR or VR headset 230, such as manual controls which can beactuated by a user or through sensor-based detection of user gestures).The host device 210 may be, for example, a computing device such as apersonal computer (PC), laptop, tablet, smartphone, gaming console,server, or another type of device which is able to communicate with theAR or VR headset 230 and execute processing tasks in connectiontherewith.

In an exemplary embodiment, the AR or VR headset 230 may include a touchsensor and/or a fingerprint sensor, each having a respective processingsystem as discussed above with respect to FIGS. 1A-1B. A combinedprocessing system which controls both a touch sensor and a fingerprintsensor may also be utilized. The processing system(s) may be implementedwithin the AR or VR headset 230 and may be in communication with thehost device 210. Alternatively, the processing system(s) may beimplemented within the host device 210.

FIG. 2C is a block diagram depicting yet another exemplary AR or VRenvironment 200 c in which exemplary embodiments of the presentdisclosure may be utilized. The environment 200 c includes, for example,a host device 210, one or more handheld user input device(s) 220, and anAR or VR headset 230 (e.g., in the form of goggles worn on a user's headwhich displays VR or AR information to the user). The handheld userinput device(s) 220 may include a single user input device to be held bythe user in one hand or two user input devices to be held by the user inboth hands. The handheld user input device(s) 220 may be in the form of,for example, remote control devices, gaming controllers, or any othershape suitable for being held by a user's hand, and may include buttonsor other inputs (such as touch-sensitive surfaces) disposed in variouspositions thereon. The host device 210 may be, for example, a computingdevice such as a personal computer (PC), laptop, tablet, smartphone,gaming console, server, or another type of device which is able tocommunicate with the display device 230 and the handheld user inputdevice(s) 220 and execute processing tasks in connection therewith.

In exemplary embodiments, one or both of the handheld user inputdevice(s) 220 may include a touch sensor and/or a fingerprint sensor,each having a respective processing system as discussed above withrespect to FIGS. 1A-1B. A combined processing system which controls botha touch sensor and a fingerprint sensor may also be utilized. Theprocessing system(s) may be implemented within the handheld user inputdevice(s) 220 and may be in communication with the host device 210.Alternatively, the processing system(s) may be implemented within thehost device 210 or in the AR or VR headset 230.

It will be appreciated that the environments 200 a-c depicted in FIGS.2A-2C are merely examples, and that exemplary embodiments of the presentdisclosure may also be implemented in other environments. For example,exemplary embodiments of the present disclosure may also be implementedin connection with a non-VR and non-AR system in which a handheld userinput device is used in connection with a non-touch display screen, andthe non-touch display screen may be implemented in the handheld userinput device and/or on a separate display device. To provide yet anotherexample, the handheld user input device may be replaced with a userinput device which is not necessarily handheld.

FIG. 3 is a schematic diagram illustrating an integrated scrollwheel andFPS user input apparatus 300 in an exemplary embodiment. The integratedscrollwheel and FPS user input apparatus 300 includes a scrollwheel 310,a fingerprint sensor (FPS) 320, and optionally a tactile ridge 330, andmay be integrated in the AR or VR headset 230 shown in FIGS. 2A-2C(e.g., based on being embedded into a side of the headset) and/or in thehandheld user input device(s) shown in FIG. 2C. It will be appreciatedthat the AR or VR headset 230 and/or the handheld user input device(s)220 may also include other buttons or input elements apart from what isdepicted in FIG. 3 .

The scrollwheel 310 may be, for example, a capacitive touch sensor, anoptical touch sensor, or a mechanical sensor. The scrollwheel 310 isconfigured to detect a user's finger (e.g., the user's thumb) being inproximity with or in contact with the scrollwheel 310 and, based oninput from the scrollwheel 310, a corresponding processing system may beable to resolve one-dimensional (1D) motion with respect to whether theuser's finger moves in a clockwise or counterclockwise direction aroundthe scrollwheel (e.g., to provide up/down or left/right scrollingnavigation in connection with a display), as well as the speed at whichthe user's finger is moving (e.g., to provide relatively slower orfaster scrolling navigation in connection with the display). In certainexemplary embodiments, the scrollwheel 310 may also be configured todetect two-dimensional (2D) motion such that, based on detected inputfrom the scrollwheel 310, a corresponding processing system may be ableto distinguish between clockwise/counterclockwise motions and othertypes of motions.

In the exemplary embodiment depicted in FIG. 3 , the scrollwheel 310 isshaped as a wheel (i.e., two concentric circles establishing an outerboundary and an inner boundary), but in other embodiments, thescrollwheel 310 may have a different shape. For example, in somealternative embodiments, the scrollwheel 310 may have a non-circularshape or may be replaced with a rectangular scrollbar.

The FPS 320 may be, for example, a capacitive FPS, an optical FPS, or anultrasonic FPS. The FPS 320 is configured to detect features of a user'sfingerprint such that a corresponding processing system may authenticatea user based on the detected input from the FPS 320. The FPS 320 mayalso be force-sensitive such that the FPS 320 is configured to detect anamount of force with which a user's finger is pressing on the FPS 320,and the corresponding processing system may register a button pressaction based on the amount of force being above a certain threshold.Alternatively, the FPS 320 may be integrated with a button, with theprocessing system registering a button press action based on the buttonbeing depressed. In yet another alternative embodiment, a processingsystem may register a button press action based on detecting thepresence of a user's finger on the FPS 320.

A tactile ridge 330 is optionally disposed between the FPS 320 and thescrollwheel 310 such that a user is able to feel the inner boundary ofthe scrollwheel with the user's finger. The tactile ridge 330 may helpto guide the user during a scrolling motion and avoid unintentionallytouching the FPS 320 while trying to scroll. The tactile ridge 330 mayalso help the user to locate the FPS 320 when the user needs to performan authentication operation and/or a button press using the FPS 320. Thetactile ridge 330 may be, for example, a relatively thin raised portionof a surface of the handheld user input device disposed between thesensing region of the FPS 320 and the sensing region of the scrollwheel.The tactile ride 330 may also include, for example, a textured pattern.

In certain exemplary embodiments, the FPS 320 and the scrollwheel 310may have separate respective controllers or may have an integratedcontroller which controls both the FPS 320 and the scrollwheel 310. Inan exemplary embodiment, the scrollwheel 310 may be implemented as onesensor having a built-in processor or application-specific integratedcircuit (ASIC), and the FPS 320 may be implemented as another sensorhaving another built-in processor or ASIC, whereby both processors/ASICscommunicate separately with an AR or VR headset and/or a host device. Inanother exemplary embodiment, the scrollwheel 310 and the FPS 320 havingseparate respective processors/ASICs may be logically combined into asingle input device which communicates through only one of the twoprocessors/ASICs.

Regardless of the specific configuration of the processor/ASIC structureof the scrollwheel 310 and the FPS 320, as discussed above, thescrollwheel 310 and FPS 320 may both be considered as being part of anintegrated scrollwheel and FPS user input apparatus 300, and theintegrated scrollwheel and FPS user input apparatus 300 may be embeddedinto an AR or VR headset. In an exemplary embodiment, the integratedscrollwheel and FPS user input apparatus 300 reports to a processor ofthe AR or VR headset, and the AR or VR headset may in turn report to ahost device or be self-contained.

In an exemplary embodiment, the integrated scrollwheel and FPS userinput apparatus 300 depicted in FIG. 3 may be implemented as part of alarger touchpad device such that the scrollwheel 310 and the FPS 320 areboth parts of the larger touchpad device.

FIGS. 4A-4B depict exemplary manners of utilizing the integratedscrollwheel and FPS user input apparatus depicted in FIG. 3 . The leftside 400 a of FIG. 4A depicts an exemplary clockwise motion performed bya user's thumb 401 on the scrollwheel, and the right side 400 b of FIG.4A depicts an exemplary corresponding menu that is displayed to the userthrough a display device. For example, based on the scrollwheeldetecting the clockwise motion shown on the left side 400 a, the hostdevice recognizes the clockwise motion as being a downward menunavigation command, and the host device adjusts a user interfacedisplayed to the user accordingly as shown on the right side 400 b ofFIG. 4A—i.e., by moving a current menu selection downwards by an amountbased on the distance and/or the speed of the clockwise motion. In thisexample, the user has scrolled down from “Menu Option 1” to “Menu Option7” based on the clockwise movement of the user's thumb on thescrollwheel, and the user could similarly navigate back to “Menu Option1” to “Menu Option 7” with a similar counterclockwise movement of theuser's thumb on the scrollwheel.

With “Menu Option 7” as the current menu selection, the user may thenprovide an activation or confirmation input with respect to “Menu Option7,” for example, by placing the user's finger over the FPS, by pressingdown on the FPS, by pressing down on a button integrated with the FPS,or by pressing some other button on a user input device or a headset.And in case “Menu Option 7” corresponds to an operation which utilizesbiometric authentication (e.g., in case executing the functionalityassociated with “Menu Option 7” corresponds to a financial transaction,an action requiring the user to log-in, or some other action for whichenhanced security may be beneficial), the user may then hold his or herthumb over the FPS as shown in the left side 400 c of FIG. 4B. As theuser is being authenticated via the FPS, the menu interface shown in theright side 400 d may include a pop-up notification which shows that theauthentication is occurring. Then, based on the authentication beingsuccessful, the operation associated with “Menu Option 7” may beexecuted (e.g., an in-app or in-game purchase may be completed, the usermay be logged-in to access certain protected functionality,

It can thus be seen from FIGS. 4A-4B that exemplary embodiments of thepresent disclosure provide an integrated scrollwheel and FPS user inputapparatus which improves user experience, especially in AR or VRsystems, by providing seamless authentication together with navigation.Based on the scrollwheel and FPS sensing regions being proximate to oneanother (e.g. with the scrollwheel surrounding the FPS as shown in FIG.3 ), the user is able to quickly and intuitively transition from menunavigation to authentication and back to menu navigation without needingto look at the user input apparatus. The tactile ridge on the innerboundary of the scrollwheel further helps to guide the user and helpsthe user to easily locate the FPS.

It will be appreciated that the menu interfaces shown in parts 400 b and400 d of FIGS. 4A-4B are merely exemplary, and that various other typesof interfaces may also utilize the integrated scrollwheel and FPS userinput apparatus shown in FIG. 3 . For example, the integratedscrollwheel and FPS user input apparatus may also be used to navigatebetween a plurality of icons (corresponding to respective applications)arranged in a line or in a grid, as well as for authenticating a user inconnection with logging to a respective application selected by theuser.

FIG. 5 is a flowchart depicting an exemplary process for utilizing anintegrated scrollwheel and FPS user input apparatus in accordance withan exemplary embodiment.

At stage 501, a user interface is displayed to a user. For example, inthe environment of FIG. 2 , a host device may generate displayinformation to be displayed on a VR or AR display device, or in otherexemplary environments, a host device may generate display informationto be displayed on other types of displays, such as a conventionalnon-touch display screen. The user interface displayed to the user mayinclude a plurality of selectable items.

At stage 503, navigation input is received from a user via an integratedscrollwheel and FPS user input apparatus, for example, via thescrollwheel depicted in FIG. 3 . The scrollwheel may detect a movementof a user's finger (such as the user's thumb) in a certain manner (suchas in a clockwise or counterclockwise rotational manner). Based on themovement detected via the scrollwheel, the host device may update thedisplay in a manner corresponding to the detected movement, for example,by adjusting a current selection in the displayed user interface (e.g.,adjusting a current menu selection as shown in FIG. 4A). Otheradjustments based on the movement detected via the scrollwheel mayinclude for example, adjustment of a volume control or other systemsetting, adjustment of a current view, or other adjustments which mayintuitively correspond to a clockwise or counterclockwise rotationalmovement of a user's finger.

It will be appreciated that, as discussed above, the host device may beseparate from a display device and a user input device (e.g., as shownin FIG. 2C), or the host device may be integrated with the displaydevice and may include a user input apparatus (e.g., as shown in FIG.2A). For example, in one exemplary implementation, the device comprisingthe display may also comprise a processor for generating the displayinformation, and thereby fulfills the roles of both the host device andthe display device. In another exemplary implementation, a single devicemay fulfill the roles of the host device, the display device, and theuser input device(s) as shown in FIG. 2A.

At stage 505, an activation input is received for a currently selecteditem in the displayed user interface. For example, the activation inputmay be the user pressing down on a button on a user input device, suchas a button integrated with the FPS of an integrated scrollwheel and FPSuser input apparatus, a button separately disposed on another part ofthe user input device, or a button separately disposed on another userinput device. In another example, the activation input may be the userpressing down on the FPS of an integrated scrollwheel and FPS user inputapparatus, wherein the FPS is configured to detect an amount of pressingforce and a corresponding processing system is configured to detect theactivation input based on the amount of pressing force being greaterthan a threshold. In yet another example, the activation input may bethe detection of the presence of the user's finger on the FPS of anintegrated scrollwheel and FPS user input apparatus.

In certain embodiments, such as when the activation input is based ondetection of the presence of the user's finger, the system maydistinguish between a slide-over motion from the scrollwheel to the FPS(which might correspond to an inadvertent touching of the FPS) versus ofa lift-up motion off the scrollwheel followed by a place-down motiononto the FPS (which is more likely to correspond to a deliberatetouching of the FPS). In case a lift-up motion off the scrollwheel isfollowed by a place-down motion on to the FPS, the processing system mayregister the activation input immediately upon detecting the placementof the finger onto the FPS. In case a slide-over motion from thescrollwheel to the FPS is detected, additional measures may be employedbefore registering the activation input—for example, the processingsystem may wait until the user's finger has remained on (or remainedstationary on) the FPS for at least a predetermined amount of timebefore registering the activation input, or the processing system mayrequire an additional lift-and-place motion on the FPS (i.e., lift-upfrom the FPS and place-down on the FPS) before registering theactivation input (in which case a corresponding prompt may be displayedto the user to instruct the user to perform the lift-and-place motion onthe FPS to provide the activation input).

In an exemplary embodiment, to detect the slide-over motion, thescrollwheel may be configured as a 2D sensor capable of not onlydetecting clockwise and counterclockwise motions on the scrollwheel, butalso capable of detecting a diagonal sliding motion towards the centerof the scrollwheel. In yet another exemplary embodiment, to detect theslide-over motion, the scrollwheel and FPS may share a processing systemconfigured for controlling both the scrollwheel and FPS such that atransition from the scrollwheel and FPS is detectable by the sharedprocessing system. In yet another exemplary embodiment, to detect theslide-over motion, the tactile ridge of the integrated scrollwheel andFPS user input apparatus may further be configured with one or moresensing electrodes such that a slide-over motion may be detected basedon, for example, detecting a transition of the finger from beingdisposed on the scrollwheel to being disposed on the FPS in a mannerwhere the finger passes through a state where the finger issimultaneously in contact with the scrollwheel, the FPS, and the tactileridge.

In certain exemplary embodiments, the reception of an activation inputat stage 505 corresponds to selection of a menu item. In certainsituations, such as when navigating from a menu to a sub-menu andopening up the sub-menu based on selection of a menu item in the basemenu, this may result in execution of an operation that does not requirebiometric authentication such that stages 507-511 would not be needed.In other situations, such as when the activation of the selected menuitem results in a financial transaction or a log-in operation or someother operation benefitting from authentication, stages 507-511 areperformed.

At stage 507, a biometric input is received via the FPS of theintegrated scrollwheel and FPS user input apparatus. In certainexemplary implementations, the scanning of the sensing region of the FPSat stage 507 is performed in combination with receiving the activationinput. For example, once the user presses down on a button integratedwith the FPS or places a finger on the FPS for activation, the FPSscanning is automatically triggered and begins such that stages 505 and507 can be thought of as being carried out together. In other exemplaryimplementations, the scanning of the sensing region is implemented as aseparate step whereby the user is required to first touch or press theFPS to provide the activation input, and then prompted to lift and placethe finger again for authentication.

At stage 509, a processing system associated with the FPS of theintegrated scrollwheel and FPS user input apparatus uses the receivedbiometric input (i.e., the user's fingerprint) to performauthentication, for example, by comparing the received biometric inputto stored templates corresponding to one or more pre-enrolled authorizedusers. It will be appreciated that the part of the processing systemwhich performs authorization may be implemented in the integratedscrollwheel and FPS user input apparatus or in a host device incommunication with the integrated scrollwheel and FPS user inputapparatus or in a display device or AR or VR headset in communicationwith the integrated scrollwheel and FPS user input apparatus.

At stage 511, the processing system (which may include a host deviceseparate from the integrated scrollwheel and FPS user input apparatus)executes an operation in response to successful authentication. Forexample, the processing system may carry out a financial transaction,perform a log-in operation, or perform some other operation after theuser is authenticated. In case the authentication fails, the processingsystem may block the operation that was requested by the user, and mayfurther request that the user make another authentication attempt.

In certain situations, it may be advantageous to provide for continuousor periodic authentication of a user who is logged-in to a system or anapplication. For example, if the user is playing a game, to avoidcheating, the system or the game application may wish to continuously orperiodically verify that the logged-in user is the user who is actuallyplaying (as opposed to one user logging in and then having some otheruser play for the logged-in user). Similarly, if the user is using asensitive network application or system, to prevent fraud and/orsecurity breaches, the network application or system may wish tocontinuously or periodically verify that the logged-in user is still theuser which is using the network application or system. In thesecontinuous or periodic authentication scenarios, a user of an integratedscrollwheel and FPS user input apparatus in accordance with exemplaryembodiments of the disclosure may be instructed (e.g., via a display oran instruction manual) to rest the user's finger on the FPS of theintegrated scrollwheel and FPS user input apparatus while thescrollwheel is not in use. The FPS of the integrated scrollwheel and FPSuser input apparatus may then perform continuous or periodic scans ofthe user's fingerprint to provide for continuous or periodicauthentication.

All references, including publications, patent applications, andpatents, cited herein are hereby incorporated by reference to the sameextent as if each reference were individually and specifically indicatedto be incorporated by reference and were set forth in its entiretyherein.

The use of the terms “a” and “an” and “the” and “at least one” andsimilar referents in the context of describing the invention (especiallyin the context of the following claims) are to be construed to coverboth the singular and the plural, unless otherwise indicated herein orclearly contradicted by context. The use of the term “at least one”followed by a list of one or more items (for example, “at least one of Aand B”) is to be construed to mean one item selected from the listeditems (A or B) or any combination of two or more of the listed items (Aand B), unless otherwise indicated herein or clearly contradicted bycontext. The terms “comprising,” “having,” “including,” and “containing”are to be construed as open-ended terms (i.e., meaning “including, butnot limited to,”) unless otherwise noted. Recitation of ranges of valuesherein are merely intended to serve as a shorthand method of referringindividually to each separate value falling within the range, unlessotherwise indicated herein, and each separate value is incorporated intothe specification as if it were individually recited herein. All methodsdescribed herein can be performed in any suitable order unless otherwiseindicated herein or otherwise clearly contradicted by context. The useof any and all examples, or exemplary language (e.g., “such as”)provided herein, is intended merely to better illuminate the inventionand does not pose a limitation on the scope of the invention unlessotherwise claimed. No language in the specification should be construedas indicating any non-claimed element as essential to the practice ofthe invention.

Preferred embodiments of this invention are described herein, includingthe best mode known to the inventors for carrying out the invention.Variations of those preferred embodiments may become apparent to thoseof ordinary skill in the art upon reading the foregoing description. Theinventors expect skilled artisans to employ such variations asappropriate, and the inventors intend for the invention to be practicedotherwise than as specifically described herein. Accordingly, thisinvention includes all modifications and equivalents of the subjectmatter recited in the claims appended hereto as permitted by applicablelaw. Moreover, any combination of the above-described elements in allpossible variations thereof is encompassed by the invention unlessotherwise indicated herein or otherwise clearly contradicted by context.

1. A virtual reality (VR) or augmented reality (AR) system, comprising:a display configured to display a user interface to a user of the VR orAR system; an integrated scrollwheel and fingerprint sensor (FPS) userinput apparatus, comprising a scrollwheel configured to detect arotational navigation input from the user and an FPS configured todetect a biometric input from the user; and a processing systemconfigured to: receive the rotational navigation input via thescrollwheel of the integrated scrollwheel and FPS user input apparatus;update a displayed user interface on the display based on the receivedrotational navigation input, wherein updating the displayed userinterface comprises updating a user selection on a displayed menu;receive an activation input for the updated user selection on thedisplayed menu via the FPS of the integrated scrollwheel and FPS userinput apparatus, wherein the activation input is a detected biometricinput; perform authentication based on the detected biometric input; andexecute an operation corresponding to the updated user selection on thedisplayed menu in response to successful authentication; wherein theprocessing system is further configured to: receive another activationinput for another user selection on the displayed menu via the FPS ofthe integrated scrollwheel and FPS user input apparatus; and executeanother operation corresponding to the another user selection on thedisplayed menu in response to the another activation input withoutperforming authorization for the another operation.
 2. (canceled)
 3. TheVR or AR system according to claim 1, wherein performing authenticationbased on the detected biometric input comprises determining whether thedetected biometric input matches a stored template of a pre-enrolledauthorized user.
 4. (canceled)
 5. The VR or AR system according to claim1, wherein the integrated scrollwheel and FPS user input apparatus andthe display are both physically integrated into a same housing.
 6. TheVR or AR system according to claim 1, wherein the integrated scrollwheeland FPS user input apparatus and the display are both physicallyintegrated into a housing of a VR or AR headset.
 7. The VR or AR systemaccording to claim 1, wherein the integrated scrollwheel and FPS userinput apparatus and the display are physically integrated into separatehousings of separate devices.
 8. A method for navigation, comprising:displaying, by a display, a user interface to a user; receiving, by aprocessing system, via a scrollwheel of an integrated scrollwheel andfingerprint sensor (FPS) user input apparatus, a rotational navigationinput; updating, on the display, the displayed user interface based onthe received rotational navigation input, wherein updating the displayeduser interface comprises updating a user selection on a displayed menu;receiving, by the processing system, an activation input for the updateduser selection on the displayed menu, wherein the activation input is abiometric input detected via an FPS of the integrated scrollwheel andFPS user input apparatus; performing, by the processing system,authentication based on the detected biometric input; and executing, bythe processing system, an operation corresponding to the updated userselection on the displayed menu in response to successfulauthentication; wherein the method further comprises: receiving, by theprocessing system, another activation input for another user selectionon the displayed menu via the FPS of the integrated scrollwheel and FPSuser input apparatus; and executing, by the processing system, anotheroperation corresponding to the another user selection on the displayedmenu in response to the another activation input without performingauthorization for the another operation. 9-10. (canceled)
 11. The methodaccording to claim 8, wherein performing authentication based on thedetected biometric input comprises determining whether the detectedbiometric input matches a stored template of a pre-enrolled authorizeduser.
 12. (canceled)
 13. The method according to claim 8, wherein theintegrated scrollwheel and FPS user input apparatus and the display areboth physically integrated into a same housing.
 14. The method accordingto claim 8, wherein the integrated scrollwheel and FPS user inputapparatus and the display are both physically integrated into a housingof a VR or AR headset.
 15. The method according to claim 8, wherein theintegrated scrollwheel and FPS user input apparatus and the display arephysically integrated into separate housings of separate devices. 16-20.(canceled)
 21. One or more non-transitory computer-readable mediumshaving processor-executable instructions stored thereon for navigation,wherein the processor-executable instructions, when executed,facilitate: displaying, by a display, a user interface to a user;receiving, by a processing system, via a scrollwheel of an integratedscrollwheel and fingerprint sensor (FPS) user input apparatus, arotational navigation input; updating, on the display, the displayeduser interface based on the received rotational navigation input,wherein updating the displayed user interface comprises updating a userselection on a displayed menu; receiving, by the processing system, anactivation input for the updated user selection on the displayed menu,wherein the activation input is a biometric input detected via an FPS ofthe integrated scrollwheel and FPS user input apparatus; performing, bythe processing system, authentication based on the detected biometricinput; and executing, by the processing system, an operationcorresponding to the updated user selection on the displayed menu inresponse to successful authentication; wherein the processor-executableinstructions, when executed, further facilitate: receiving, by theprocessing system, another activation input for another user selectionon the displayed menu via the FPS of the integrated scrollwheel and FPSuser input apparatus; and executing, by the processing system, anotheroperation corresponding to the another user selection on the displayedmenu in response to the another activation input without performingauthorization for the another operation.
 22. The one or morenon-transitory computer-readable mediums according to claim 21, whereinperforming authentication based on the detected biometric inputcomprises determining whether the detected biometric input matches astored template of a pre-enrolled authorized user.
 23. The one or morenon-transitory computer-readable mediums according to claim 21, whereinthe integrated scrollwheel and FPS user input apparatus and the displayare both physically integrated into a same housing.
 24. The one or morenon-transitory computer-readable mediums according to claim 21, whereinthe integrated scrollwheel and FPS user input apparatus and the displayare both physically integrated into a housing of a VR or AR headset. 25.The one or more non-transitory computer-readable mediums according toclaim 21, wherein the integrated scrollwheel and FPS user inputapparatus and the display are physically integrated into separatehousings of separate devices.